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ABSTRACT 

Although administrative health care databases have long been used to evaluate adverse drug effects, responses to 
drug safety signals have been slow and uncoordinated. We describe the establishment of the Canadian Network for 
Observational Drug Effect Studies (CNODES), a collaborating centre of the Drug Safety and Effectiveness Network 
(DSEN). CNODES is a distributed network of investigators and linked databases in British Columbia, Alberta, Sas- 
katchewan, Manitoba, Ontario, Quebec and Nova Scotia. Principles of operation are as follows: (1) research ques- 
tions are prioritized by the coordinating office of DSEN; (2) the linked data stay within the provinces; (3) for each 
question, a study team formulates a detailed protocol enabling consistent analyses in each province; (4) analyses 
are "blind" to results obtained elsewhere; (5) protocol deviations are permitted for technical reasons only; (6) ana- 
lyses using multivariable methods are lodged centrally with a methods team, which is responsible for combining 
the results to provide a summary estimate of effect. These procedures are designed to achieve high internal validity 
of risk estimates and to eliminate the possibility of selective reporting of analyses or outcomes. The value of a coor- 
dinated multi-provincial approach is illustrated by projects studying acute renal injury with high-potency statins, 
community-acquired pneumonia with proton pump inhibitors, and hyperglycemic emergencies with antipsychotic 
drugs. CNODES is an academically based distributed network of Canadian researchers and data centres with a 
commitment to rapid and sophisticated analysis of emerging drug safety signals in study populations totalling over 
40 million. 
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► The need for drug safety research using an 
epidemiological approach has been clearly understood 
for decades. 1 ' 2 Prescription medications remain one of 
the most common causes of severe adverse reactions in 
clinical medicine, accounting for an estimated 1800 to 
10 000 deaths annually in Canada. 3 ' 4 

Canadian population health databases have been 
used to assess the risks and benefits of non-steroidal 
anti-inflammatory drugs (NSAIDs), beta-agonist in- 
halers for the treatment of asthma, anti-psychotic 
drugs, gastric-acid suppressants and many other phar- 
maceutical therapies. 5-8 A population-based approach 
is particularly important for less frequent, severe or 
long-term adverse effects that cannot be detected by 
the randomized controlled trials required for initial 
drug approval. Such trials are not powered for rare 
outcomes, exclude vulnerable populations and do not 
provide sufficient follow-up for the quantification of 
long-term effects. 9 

Recent experience concerning the cardiovascular 
effects of cyclo-oxygenase-2 inhibitors and thiazoli- 
dinediones demonstrates the need to rapidly detect and 
confirm low relative risks, in the order of 1.2-1.5, to 
be able to distinguish between individual members of 
drug classes with respect to their associated risks and 
to identify clinical factors that increase the risk of ad- 
verse drug effects. 1011 This requires very large sample 
sizes, which can be achieved only through the use of 
population databases. 

To date, such research has suffered from a lack of co- 
ordination. For example, investigations of the adverse 
cardiovascular effects of rofecoxib were conducted by 
separate teams of researchers using databases in On- 
tario, Quebec and Saskatchewan. 12-14 The time taken 
to respond to the first report on safety concerns, pub- 
lished in November 2000, 15 ranged from 3 to 9 years— 
an excessive period, considering the potential threat to 
public health posed by a widely used drug. Investiga- 
tors used different approaches in designing their stud- 
ies and analyzing their results, discrepant results were 
obtained, and individual risk estimates were imprecise. 
These studies were performed by small academic 
groups working within a system of competitive funding 
that rewards individual rather than collective effort. 

The challenges are to organize sufficient financial 
and human resources, to coordinate responses to safety 
signals, to standardize methodological approaches and 
to obtain rapid access to data sets that are large enough 
to give precise estimates of risk. The Canadian Network 
for Observational Drug Effect Studies (CNODES), an 
investigator-led, multi-provincial distributed network 



of data repositories and researchers, has been estab- 
lished to do this. 

The development of CNODES 

CNODES (www.cnodes.ca) is part of the Drug Safety 
and Effectiveness Network (DSEN), a joint initiative of 
Health Canada and the Canadian Institutes of Health Re- 
search (CIHR) (www.cihr-irsc.gc.ca/e/40269.html). The 
principal aim of CNODES is to use collaborative, popu- 
lation-based approaches to obtain rapid answers to 
questions about drug safety and effectiveness. Funding 
for the CNODES infrastructure was granted in Janu- 
ary 2011 on the basis of a single, directed, internation- 
ally refereed application to CIHR, with representation 
from 7 provinces (British Columbia, Alberta, Saskatch- 
ewan, Manitoba, Ontario, Quebec and Nova Scotia). 
The application added a mechanism for accessing data 
from the United Kingdom Clinical Practice Research 
Datalink (CPRD)— previously known as the General 
Practice Research Database— in view of its size and 
the direct and rapid access it provides to comprehen- 
sive data, including on drugs marketed in the United 
Kingdom before they are licensed in Canada. 16 Because 
CPRD is a compilation of electronic health records, it 
enables adjustment for potential confounders that are 
not routinely captured in administrative records (e.g., 
smoking). Work within CNODES began in March 2011. 

Database structure. Legal and privacy concerns made 
it unfeasible to pool data from multiple provinces in a 
single, central repository. There was a strong prefer- 
ence for a distributed network, as this would enable the 
data to remain and be analyzed "in situ," meaning that 
local regulations concerning data access and approval 
processes for research would be respected, and local 
knowledge, expertise and analytical capacity would 
be valued and supported. The main disadvantages are 
that those same features can lead to delays in access- 
ing data or obtaining approvals for individual studies. 
Some barriers to timely data access persist in British 
Columbia and Quebec. 

CNODES data sources are summarized in Table 1. 
The provincial databases vary in their capacity to an- 
swer different questions. For example, if a question 
concerns a drug used primarily in patients under the 
age of 65, this can be studied comprehensively in only 3 
provinces (British Columbia, Saskatchewan and Mani- 
toba) plus the CPRD. 

Governance. The CNODES governance structure is 
depicted in Figure 1. Prospective research questions 
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regarding safety and effectiveness of drugs marketed 
in Canada come from Health Canada and are directed 
to the DSEN Coordinating Office. Federal and prov- 
incial drug plans may also refer questions to Health 
Canada. Before consideration by CNODES, the ques- 
tions are assessed and prioritized by a DSEN Scientific 
Advisory Committee, which includes staff from Health 
Canada and provincial drug plans, as well as experts in 
pharmacy and pharmaco-epidemiology). The CNODES 
Database Team, comprising the provincial leads, is re- 
sponsible for ensuring access to the linked provincial 
data sets and maintaining analytical capacity at each 
site. If a study is feasible within CNODES, the Database 
Team selects a Project Team composed of nominees 
from each province with relevant methodological and 
content expertise and a liaison from the Methods Team. 
Data analysts are included at all stages of project plan- 
ning and discussion. The 15 to 20 members of the Pro- 
ject Team develop the common scientific and analytical 
protocols, enable the sharing of algorithms and statis- 
tical analysis software, organize local approvals for the 
research and arrange for the analyses to be conducted 



in each database. The 7 members of the Methods Team 
are statisticians, biostatisticians or epidemiologists. 
They provide methodological leadership, assist with 
the development of the scientific and analytical proto- 
cols for each project and conduct meta-analyses of the 
results across databases. The Methods Team also re- 
ports on bias, precision and validity of results, and is 
charged with developing new methods for the design 
and analysis of multiple observational studies. 

Methods. Linked prescription drug and outcome data 
are analyzed separately in each province using a range 
of multivariable techniques; the results from the prov- 
inces and the CPRD are combined to give a summary 
estimate. In the studies conducted to date (see Table 2), 
a range of methods and tools have been used, includ- 
ing high-dimensional propensity scores to generate 
and compare exposed and reference cohorts and nest- 
ed case-control analyses with propensity score adjust- 
ment. A study of the association between proton pump 
inhibitors (PPIs) and pneumonia employed a highly re- 
stricted cohort, selected from new users of NSAIDs, to 



Table 1 

Overview of CNODES Canadian provincial databases and the CPRD (as of June 201 2)* 



Site 


Total 

population 

(000s) 


Prescription drug 
data: patient 
groups covered; 
period covered 


Frequency 
of updates 


Vaccine data 


Emergency 
department 
encounters 


Outpatient 
laboratory 
data 


Cancer 
registry 


Time to 
access data 


British 
Columbia 


4573 


All; 

from 1 996 


Weekly 


Partial 


No 


No 


No 


Weeks 


Alberta 


3779 


Age > 65; 
from 1 994 


Monthly 


Yes 


Yes 


No 


No 


Days 


Saskatchewan 


1 058 


All; 

from 1 996 


Quarterly 


No 


No 


No 


No 


Days 


Manitoba 


1 251 


All; 

from 1 995 


Quarterly 


Yes 


Yes 


Partial 

(public health) 


Yes 


Days 


Ontario 


13373 


Age > 65, 
receiving social 
assistance; 
from 1 997 


Bimonthly 


No 


Yes 


No 


Yes 


Days 


Quebec 


7980 


Age > 65, 
receiving social 
assistance; 
from 1 983 


Monthly 


No 


Yes 


No 


Yes 


Months 


Nova Scotia 


945 


Age > 65, 
receiving social 
assistance / Family 
Pharmacare; 
from 1 989 


Quarterly 


No 


Yes 


No 


Yes 


Days 


CPRD 


11 829 


AII.- 


Monthly 


Yes 


No 


Yes 


Yes 


Immediate 



from 1 988 

CNODES = Canadian Network for Observational Drug Effect Studies, CPRD = [United Kingdom] Clinical Practice Research Datalink 
*AII sites have access to data on demographic characteristics, vital statistics, dispensed outpatient prescriptions, physician encounters and hospital 
admissions. 
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minimize confounding by indication (esophagitis) and 
reverse causality (PPIs being prescribed for chest pain 
from pneumonia) (Table 2). This was possible only be- 
cause of the large source populations. 

Measures to reduce bias. The complete coverage of 
provincial populations and subpopulations minimizes 
selection biases. The initial screening of study ques- 
tions ensures their appropriateness for evaluation 
using observational methods and administrative data. 
There is strong clinical and methodological input into 
the design of scientific and analytical protocols. The 
standardization of these reduces variation in results 
arising from study design and analysis. We use com- 
mon exposure, covariable and outcome definitions and 
make maximum use of shared common, tested, stat- 
istical analysis (SAS) code. Sophisticated analytical 
techniques reduce confounding (see examples in the 
previous section on methods). All research protocols are 
logged at the CNODES and DSEN coordinating offices, 



and will be registered with www.Clinicaltrials.gov. 
All study outcomes and analyses are pre-specified, 
and site-specific protocol deviations are permitted for 
technical reasons only. Each participating site lodges 
results in a secure drop box without knowing the re- 
sults obtained at other sites. The Project Team and 
Steering Committee jointly review and interpret re- 
sults. All reports are reviewed and approved by a Pub- 
lications Committee, which currently consists of the 
CNODES executive but will include two additional ex- 
ternal members in future. These procedures should re- 
duce publication and reporting biases. 17,18 All CNODES 
researchers are required to make a full disclosure 
of their relationships with life sciences companies. 
Open-ended relationships, such as stock holding and 
membership in speakers' bureaus are discouraged; 
researchers with such relationships cannot take pri- 
mary responsibility for leading specific CNODES re- 
search projects, including developing study protocols 
or drafting study reports. 



DSEN 
query 



DSEN 
Coordinating 
Office 



Feasibility assessment and prioritization 
by DSEN Scientific Advisory Committee 



BC 



AB 



SK 



MB 



ON 



QC 



NS 



CPRD 



Methods 
Team 



Database 
Team 



Selection of Project Team; 
appointment of Project 
Team lead, Project Team 
liaisons and Methods 
Team liaison 



CNODES 
Coordinating 
Centre and 

Steering 
Committee 




Knowledge 
Translation 
Team 



Design/implement 
protocol 



Lodge results 



Review and synthesize 
data (Methods Team) 



Prepare reports 



Review by CNODES 
Publications 
Committee 



Submit to 
DSEN Coordinating 
Office, provincial 
ministries/data 
custodians and 
for peer-reviewed 
publication 



Figure 1 

How questions are prioritized and answered by the Canadian Network for Observational Drug Effect Studies (CNODES). 

CPRD = [United Kingdom] Clinical Practice Research Datalink 
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Studies undertaken to date. CNODES has begun 5 
studies since March 2011. Analyses were completed 
within 12 months for 3 of these studies (Table 2). In- 
itial study questions were designed to use data from all 
member provinces. The details of the studies illustrate 
the capacity of the network to assemble very large co- 
horts, to agree on standardized analytical protocols 
and to carry out suitable adjustments simultaneously 
in multiple data sets. Early experience indicates that 
important questions can be addressed in most prov- 
inces within 4 months. Full results of studies are being 
published separately and are not provided here. Briefly, 
the results documented a small increase in the rela- 
tive risk of acute kidney injury with high- versus low- 
potency statins and no increase in the risk of community- 
acquired pneumonia with proton pump inhibitors. 19 ' 20 

Limitations. The main limitations are those of the 
data sets themselves. Eight of the 9 repositories used 
by CNODES comprise linked health administrative 
data, which generally lack information on risk factors 
such as obesity, smoking, non-prescription drugs and 
complementary therapies. 21 The CPRD database in the 
United Kingdom comprises electronic health records 
that include some of this information, 16 and we found 
no difference between the CPRD and the provincial 
administrative data in the results obtained to date. 
Databases do not include hospital drug use, and they 
capture information on drug prescribing (CPRD) or 
dispensing, rather than consumption. Controlled ob- 
servational studies are susceptible to a range of biases, 



most significantly information bias and confounding 
by indication and disease severity. 22 These are limita- 
tions of all non-randomized pharmaco-epidemiological 
studies, but reviews suggest that there is good agree- 
ment between properly designed non-randomized and 
randomized studies in the investigation of the adverse 
effects of drugs. 23 

Capacity building and knowledge translation. Train- 
ing and knowledge translation teams have been estab- 
lished to support the work of CNODES. Key objectives 
of the network are to train graduate students and ana- 
lysts, and to inform clinical practice and health care 
programs and policies. 

Discussion 

The main advantages of this multi-provincial distrib- 
uted network are the generalizability, statistical power 
and timeliness of the study results. The standardization 
of design and analyses minimizes variation. The large 
group of researchers in 7 provinces is building substan- 
tial capacity in pharmaco-epidemiology across Canada. 

In establishing CNODES we considered that a dis- 
tributed network in which data are analyzed in each 
province and outputs are combined by meta-analysis 
would be the only feasible model. This overcame the 
substantial legal and privacy obstacles to pooling the 
raw data. However, the distributed network approach 
has not guaranteed timely access to data in every prov- 
ince. There were, and still are, substantial delays in 
British Columbia and Quebec. 



Table 2 

Summary of CNODES projects completed between March 201 1 and May 2012 



Study question 

Do high-dose statins 
increase the risk of acute 
renal damage? 



Design 

Retrospective cohort study 
of high- versus low-dose 
statins 



Analysis 

Meta-analysis of site-level, 
propensity score-adjusted 
intention-to-treat and 
as-treated (case-control) 
analyses 



Data sets used in analysis 

BC, AB, SK, MB, ON, QC, NS 
plus CPRD 



Sample sizes 

646803 users of high-dose 
statins and 1 351 162 users 
of low-dose statins 



Do PPIs increase the risk 
of community- acquired 
pneumonia? 



Restricted retrospective 
cohort study (of patients 
who commenced PPI and 
NSAID therapy on the same 
day) 



Meta-analysis of site-level, 
propensity score-adjusted 
intention-to-treat analyses 



AB, SK, MB, ON, QC, NS plus 
CPRD 



93 835 PPI/NSAID users and 
4228184 non-users 



Do all second-generation 
antipsychotic drugs 
increase the risk 
of hyperglycemic 
emergencies? 



Retrospective cohort study 



Meta-analysis of site-level, 
propensity score-adjusted 
intention-to-treat and as- 
treated analyses 



Age 1 8-65: BC, SK, MB plus 
CPRD 

Age > 66: BC, AB, SK, MB, 
ON, NS plus CPRD 



215 591 aged 18-65 and 
298 580 aged > 66 



AB = Alberta, BC = British Columbia, CNODES = Canadian Network for Observational Drug Effect Studies, CPRD = [United Kingdom] Clinical Practice Research Datalink, 
MB = Manitoba, NS = Nova Scotia, NSAID = non-steroidal anti-inflammatory drug, ON = Ontario, PPI = proton pump inhibitor, QC = Quebec, SK = Saskatchewan 
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CNODES exploits the scientific principle of replica- 
tion. A single study is rarely sufficient to provide de- 
finitive information on a serious adverse drug effect 
unless it finds an extremely high risk for a drug. In 
CNODES the drug-disease association is examined in 
up to 9 databases simultaneously, and early experience 
has shown that heterogeneity in the estimates is low. 
This contrasts with the typical situation, in which stud- 
ies are conducted in a haphazard, sequential manner 
over several years before a meta-analysis is performed 
to assess the totality of the evidence. 10 In CNODES, 
meta-analyses are planned prospectively as part of 
the study protocol, providing immediate summary 
evidence. The value of such a network is illustrated by 
the study of high-dose statins and acute kidney injury, 
which explored an uncommon, serious outcome and a 
small excess risk with a widely used medication. 19 

CNODES exploits the enormous statistical power 
that results from combining provincial health data- 
bases. This discriminatory power is necessary to detect 
small but important increases in risk and to make com- 
parisons between drugs, so that benefit-harm relation- 
ships can be quantified and contrasted. Drug regulation 
is moving from a traditional "all or nothing" attitude to 
safety to a "risk management" approach throughout the 
life cycle of the drug. 24 This necessitates more rapid ac- 
cess to larger and more comprehensive data sets and 
people with the skills to analyze them. 

The strategy used to establish CNODES differs from 
that applied in the creation of Mini- Sentinel, a prom- 
inent drug safety network in the United States. Mini- 
Sentinel is a distributed network of 17 data partners 
representing a total population of 99 million. 25 Much 
development work went into the creation of data ex- 
traction tools that generate identical core data sets 
within each node. These data nodes can be queried dir- 
ectly and simultaneously from the coordinating centre 
using common code with rapid response times (days 
rather than weeks). 25 Although CNODES is not capable 
of functioning on this time scale, it has the advantages 
of complete population coverage and the academic 
and methodological capacity that is being built at each 
provincial site. This new expert network will serve the 
long-term interests of Canadians with respect to drug 
safety. 

Currently, CNODES has access to data from a total 
population of over 40 million, and many pharmaco- 
epidemiologists in Canada either already have or will 
have some connection with the enterprise. Studies that 
are independent of CNODES will continue and hopeful- 
ly increase as a result of the larger capacity engendered 



by the network. The first real test will be the ability of 
CNODES to quantify the next major drug safety signal. 
Experience suggests that we will not have to wait very 
long for that event. 

Contributors: All authors were involved in the creation of CNODES by 
contributing to the original funding application, negotiating access to 
data through their respective provincial ministries or health authorities, 
and establishing the local resources needed to undertake the work of 
CNODES. All authors contributed to the conception and design of studies, 
the acquisition of data, and the local and/or central analysis of data, and 
all assisted in drafting and/or editing the manuscript. 
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